CrowdReply: A Crowdsourced Multiple Choice Question Answering System
نویسندگان
چکیده
Crowdsourcing has applications where machine learning falls short. We describe the design, implementation, and deployment of CrowdReply, a crowdsourced multiple choice question answering (MCQA) system. We deployed an Android app to enable the audience watching “Who wants to be millionaire?” quiz-show on TV to play along on their smartphones simultaneously. The app enabled us to collect data from 1000s of users about MCQA dynamics. Our findings indicate that it is possible to aggregate the crowd’s answers to build a superplayer.
منابع مشابه
Crowdsourcing for Multiple-Choice Question Answering
We leverage crowd wisdom for multiple-choice question answering, and employ lightweight machine learning techniques to improve the aggregation accuracy of crowdsourced answers to these questions. In order to develop more effective aggregation methods and evaluate them empirically, we developed and deployed a crowdsourced system for playing the “Who wants to be a millionaire?” quiz show. Analyzi...
متن کاملارایه یک پیکره پرسش و پاسخ مذهبی در زبان فارسی
Question answering system is a field in natural language processing and information retrieval noticed by researchers in these decades. Due to a growing interest in this field of research, the need to have appropriate data sources is perceived. Most researches about developing question answering corpus area have been done in English so far, but in other languages as Persian, the lack of these co...
متن کاملCMUQA: Multiple-Choice Question Answering at NTCIR-12 QA Lab-2 Task
The first version of the UIMA-based modular automatic question answering (QA) system was developed for NTCIR11 QA Lab task[5]. The system answers multiple-choice English questions for the Japanese university entrance examinations on the subject of world history. We made improvements in the current system by adding components focused towards Source Expansion and better Semantic Understanding of ...
متن کاملsJanta: An Open Domain Question Answering System
This paper reports on the participation of the system namely “sJanta” at NII Testbeds and Community for Information Access Research Project 11 (NTCIR-11) Question Answering (QA)-Lab English sub-task. sJanta is a modular question answering system that can answer multiple choice questions given in English natural language. We use English Wikipedia as knowledge-base. At first we retrieved Wikipedi...
متن کاملBoosting Passage Retrieval through Reuse in Question Answering
Question Answering (QA) is an emerging important field in Information Retrieval. In a QA system the archive of previous questions asked from the system makes a collection full of useful factual nuggets. This paper makes an initial attempt to investigate the reuse of facts contained in the archive of previous questions to help and gain performance in answering future related factoid questions. I...
متن کامل